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ABSTRACT 


Data  from  test  runs  of  the  Surface  Effects  Ship 
XR-1D  test  craft  were  collected  and  sorted  into  four 
performance  classes  with  respect  to  the  broaching  (air 
ingestion)  phenomenon.  These  classes  included  broach, 
no-broach  and  two  intermediate  conditions.  Each  data 
point  was  characterized  by  a  twenty  parameter  measure¬ 
ment  vector  and  the  resulting  20-dimensional ,  four  class 
data  base  was  analyzed  by  the  On  Line  Pattern  Analysis 
and  Recognition  System  (OLPARS)  to  determine  the  degree 
of  separability  of  broach  from  no-broach  as  well  as  the 
measurements  most  meaningful  in  achieving  this  separa¬ 
bility.  The  results  show  that  broach  is  separable  from 
no-broach  with  low  error  and  that  no  measurement  parameter 
other  than  draft  had  any  clearly  greater  effect  on 
results  than  any  other  parameter. 


ADMINISTRATIVE  INFORMATION 

This  report  was  written  in  The  Computation,  Mathematics  and  Logistics 
Department  under  the  sponsorship  of  the  Naval  Sea  Systems  Command,  Surface 
Effect  Ship  Project,  Task  Area  Number  S0308SH001 ,  Project  S0308SII,  Program 
Element  63534N  63534N,  Job  Order  1-1824-006. 


INTRODUCTION 


Data  from  six  test  runs  of  the  Surface  Effects  Ships  XR-1D  test 
craft  were  collected  and  separated  according  to  four  performance  classes: 
normal  or  no-broach  (Zone  0),  full  broach  (Zone  3),  and  two  intermediate 
stages  of  broach  (Zone  1  and  Zone  2).  A  broach  is  caused  by  air  ingested 
into  the  water-jet  propulsion  system  of  the  craft  and  is  signalled  by  a 
sudden  decrease  in  pump  torque.  The  strip  charts  of  the  pump  torque 
measurements  taken  during  the  test  runs  were  visually  scanned  and  all 
broach  or  partial  broach  conditions  were  noted  along  with  their  time  of 
occurrence.  Recorded  data  from  the  XR-lD's  instrumentation  plus  the  test 
logs  then  became  the  basis  of  twenty  measurements  taken  for  each  broach 
and  a  certain  number  of  non-broaches.  The  resulting  four-class  data  base 
was  analyzed  with  the  On  Line  Pattern  Analysis  and  Recognition  System 
(OLPARS)  on  the  CDC  6700  computer  at  the  Computation,  Mathematics  and 
Logistics  Department  of  DTNSRDC.  This  analysis  determined  linear  separ¬ 
ations  of  broach  conditions  from  no-broach  conditions  and  identified 
parameters  most  (or  least)  effective  in  achieving  the  separation. 


DATA 


Data  were  collected  from  six  different  test  runs  of  the  XR-1D  test 
craft  in  the  summer  of  1977.  From  instrumentation  aboard  the  craft 
and  information  from  the  test  logs,  a  vector  composed  of  twenty  measure¬ 
ments  was  derived  for  each  test  point.  A  test  point  is  a  confirmed  (Zone 
1,  Zone  2,  or  Zone  3)  broach  condition  or  a  no-broach  (Zone  0)  occurring 
during  a  specifically  determined  time  interval  in  a  test  run.  The  time 
slices  were  identified  by  visually  scanning  the  readouts  of  the  pump 
torque  measurements  and  recording  the  time,  duration,  and  class  (Zone  1,2, 
or  3)  of  each  broach  occurring  during  test  operations.  A  similar  pro¬ 
cedure  was  followed  to  identify  a  number  of  no-broach  (Zone  0)  conditions. 
Zone  3  is  characterized  by  a  full  minimum  of  pump  torque  for  a  definite 
amount  of  time,  and  Zone  0  is  characterized  by  a  constant  full  torque. 
Zones  1  and  2  are  intermediate  levels.  The  distinction  between  zones, 
therefore,  is  subject  to  some  variation  resulting  from  personal  interpre¬ 
tation.  In  these  tests,  all  broach  data  points  were  measured  and  a  number 
of  no-broach  data  points,  approximately  equal  to  the  number  of  Zone  3  data 
points,  were  included  to  complete  the  data  set.  The  final  distribution  of 
test  points  (after  some  deletions  due  to  incomplete  measurements)  is  as 
follows : 


Zone 

0 

152 

point  s 

Zone 

1 

200 

point  s 

Zone 

2 

2  76 

points 

Zone 

3 

147 

points 

Total 

775 

point  s 
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The  twenty  measurement  parameters  chosen  to  characterize  the  broach 
data  points  are  listed  in  Table  1  in  the  order  in  which  they  appeared 
in  the  20-dimensional  measurement  vectors.  As  indicated  in  Table  1, 
thirteen  of  the  measurements  are  obtainable  from  on-board  instrumentation 
which  provides  analog  recordings  of  these  parameter  values.  The  actual 
test  point  value  for  each  of  these  parameters  was  obtained  by  sampling 
the  analog  signal  at  50  Hertz  during  the  test  point  interval  and  averag¬ 
ing  the  values  received.  Parameters  number  11  and  numbers  16  through 
20  were  obtained  from  test  logs,  and  parameter  number  10  was  obtained 
by  visual  observation  of  video  tapes  of  the  port  inlet  during  the  test 
point  time  slice. 

The  concern  resulting  from  previous  investigations1*  into  broach 
classification  was  that  test  data  did  not  provide  a  sufficient  number  of 
broach  data  points  to  lend  great  confidence  to  any  analytical  findings. 
Consequently,  in  this  test  series,  much  effort  was  expended  to  include 
data  for  all  broach  conditions  encountered  during  testing  and  also  to 
include  approximately  as  many  Zone  0  no-broach  data  points  as  Zone  3 
points.  The  resulting  distribution  of  data  points  does  indeed  seem  to 
represent  the  broach  conditions  well.  However,  new  questions  have  arisen 
about  the  adequacy  of  the  Zone  0  distribution,  specifically  whether  the 
relative  distribution  of  broach  points  to  no  broach  points  actually 
reflects  the  frequency  of  occurrence  of  these  states  in  the  normal  opera¬ 
tion  of  the  craft.  A  more  representative  set  of  no-broach  points  prob¬ 
ably  would  be  larger  than  the  current  set  and  would  also  contain  many 


*A  complete  listing  of  references  is  given  on  page  32. 


TABLE  1 .  MEASUREMENT  PARAMETERS  VECTOR  ORDER 


*  1.  Craft  speed  (knots) 

*  2.  Pitch  angle  (deg.) 

*  3.  Pitch  rate  (deg. /sec.) 

*  4.  Yaw  rate  (deg. /sec.) 

*  5.  Roll  angle  (deg.) 

*  6.  Roll  rate  (deg. /sec.) 

*  7.  Cushion  pressure  #3  (p.s.f.) 

*  8.  Port  inlet  velocity  ratio  (IVR) 

*  9.  Starboard  IVR 

10.  Draft  (inches) 

11.  Stern  seal  position  (inches) 

*12.  Vertical  center  of  gravity  (VCG)  acceleration  (g) 
*13.  Total  fan  air  flow  (c.f.s.) 

*14.  Port  nozzle  angle  (deg.) 

*15.  Starboard  nozzle  angle  (deg.) 

16.  Gross  weight  (lbs.) 

17.  Longitudinal  center  of  gravity  (LCG)  (ft.) 

18.  Sea  state  (0,  1 ,  2) 

19.  Heading  (l=head,  2=beam,  3=follow,  4=quarter,  5=calm) 

20.  Turn  or  straight  (1=T,  2=S) 

*  Obtained  from  on-board  analog  instrumentation 
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more  points  than  the  broach  sets.  The  effect  of  a  larger  no  broach  data 
set  on  the  results  of  this  study  is  not  known  and  depends  on  the  degree 
of  uniformity  of  the  measurement  set  during  no  broach  operation.  Accord¬ 
ingly,  future  broach  data  sets  should  have  a  greater  concentration  of 
no-broach  data  points. 

OLPARS 

2 

OLPARS  (On  Line  Pattern  Analysis  and  Recognition  System)  is  an 
interactive  computer  graphics  system  available  on  the  CPC  6700  computer 
at  the  Computation,  Mathematics  and  Logistics  Department.  It  allows  the 
user  to  examine  two-dimensional  projections  of  multidimensional  data, 
cluster  the  data,  and  draw  piecewise  linear  discriminants. 

ANALYSIS 

MEASUREMENT  EVALUATION 

The  OLPARS  analysis  of  this  broach  data  set  initially  encountered 
data  formatting  problems  which  were  avoided  by  scaling  the  broach  data 
set  to  obtain  a  uniform  order  of  magnitude  for  all  measurement  compon¬ 
ents;  specifically,  the  fan  flow  measurement  was  divided  by  100  and  the 
weight  by  1000. 

An  OLPARS  discriminant  vector  analysis  of  half  of  the  data  set 
using  all  twenty  dimensions  (measurements),  yielded  the  following  linear 
discriminant  function: 
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f  (x)  =  - 

.0159x1  +  .0162x2  + 

.0115x3  - 

. 0499x.  -  . 01 26x, 

4  5 

+ 

. 0086x,  -  . 
o 

,0120x?  + 

.6930xg  - 

.6913Xg  +  .0149xjq 

+ 

.0223xn  +  .0251Xj2 

-  .0745x13 

-  .0188x,,  -  ,0130x,c 
14  15 

+ 

.0368x^6  + 

.0733xj  ? 

-  . 1456Xj g 

-  .0053xJ9  +  .0548x2O 

-  1.5097  =  0 

where  x  =  (x^,  ....  x2q)  t^>e  20-dimensional  measurement 

vector  and  f(x)>0  for  no  broach  vectors  and  f(x)<0  for  broach  vectors. 
This  discriminant,  when  tested  against  the  remaining  half  of  the  data  set 
exhibited  the  following  classification  results: 


percentage  of  Zone  0  classed  as  broach  (Zone  0  false  alarm):  7.9% 

percentage  of  Zone  1  classed  as  broach  (Zone  1  false  alarm):  45% 

percentage  of  Zone  2  classed  as  no  broach  (Zone  2  error)  :  19.6% 

percentage  of  Zone  3  classed  as  no  broach  (Zone  3  error)  :  5.6% 


These  results  indicated  that  the  different  zones  were  at  least  partially 
separable,  and  further  analysis  was  pursued  to  improve  the  classif icat ion 
rates  and  identify  those  measurement  parameters  having  the  greatest 
effect  on  results. 

Subsequent  analysis  of  this  broach  data  set  included  the  computation 
of  histograms  for  each  parameter  in  each  broach  zone,  examination  of 
covariance  and  eigen  vector  data,  and  the  OLPARS  analysis  of  coordinate 
plots  of  pairs  of  measurement  parameters.  (See  the  accompanying  Ap¬ 
pendixes.)  Discussion  with  representatives  of  the  Surface  Effect  Ships 
Project  (Codes  PMS- 304-221,  1602,  and  1630)  about  the  physical  meaning  of 
each  parameter  and  the  test  conditions  resulted  in  reducing  the  number  of 
parameters  to  be  considered  to  fourteen  by  excluding  the  port  and  star¬ 
board  IVR's,  the  VCG  acceleration,  weight,  LCG,  and  sea  state  parameters. 
The  sea  states  in  the  data  set  were  all  0,  1,  and  2,  and  it  was  felt 
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they  varied  too  little  and  the  measurement  at  that  level  was  too  subject¬ 
ive  to  be  of  much  importance.  The  IVR's  were  judged  to  be  directly 
dependent  on  the  pump  torques  -  the  measurements  used  to  manually  isolate 
the  broaches  -  and,  as  such,  were  supplemental  effects  of  broaching 
rather  than  possible  indicators.  The  weight,  LCG,  and  VCG  were  consid¬ 
ered  to  be  dependent  and  non-controllable  and,  therefore,  not  meaningful 
for  broach  prediction.  The  fourteen  remaining  parameters,  listed  in 
Table  2,  were  then  subjected  to  a  series  of  discriminant  analysis  proce¬ 
dures  in  an  attempt  to  identify  the  hierarchy  of  sensitivity  among  the 
parameters  to  broach/no  broach  classification. 

SENSITIVITY  ANALYSIS  WITH  OLPARS 

OLPARS  allows  the  user  to  visually  examine  two-dimensional  projec¬ 
tions  of  data  in  addition  to  obtaining  computational  results.  Using 
the  set  of  fourteen  meaningful  parameters  listed  in  Table  2,  OLPARS 
can  easily  supply  a  linear  discriminant  with  associated  false  alarm  and 
error  rates.  A  reasonable  discriminant  for  this  broach  data  set  in  these 
fourteen  measurements  (dimensions)  was  found  to  be 

f ( x )  =  -  . 1465x,  +  ,0005x„  -  .5188x-  -  .0645x.  -  .1447x. 

1  2  3  4  5 

-  ,0662x^  +  .2454xy  +0.  Xg  +  O.x^  +  .3990x^q 

+  .2700xjj  +  0.  x^  “  . 05909x^2  +  .1027xj^  +  .1205xj^ 

+  0.  Xjg  +  0.  Xj^  +  0-Xi8  +  -1146xjg  -  .0126x2q 

-  8.8143  =  0 

where  x  =  (xj ,  . . .  ^(j)  *s  t'ie  20-dimensional  measurement 

vector,  as  before,  and  zero  coefficients  are  attached  to  those  measure¬ 
ment  components  not  being  considered.  Once  again  f(x)  is  positive 
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TABLE  2.  REDUCED  SET  OF  MEANINGFUL  PARAMETERS 


1.  Craft  speed  (kts) 

2.  Pitch  angle  (deg.) 

3.  Pitch  rate  (deg. /sec.) 

4.  Yaw  rate  (deg. /sec.) 

5.  Roll  angle  (deg.) 

6.  Roll  rate  (deg. /sec.) 

7.  Cushion  pressure  #3  (p.s.f.) 

10.  Draft  (inches) 

11.  Stern  seal  position  (inches) 

13.  Total  fan  air  flow  (c.f.s.) 

14.  Port  nozzle  angle  (deg.) 

13.  Starboard  nozzle  angle  (deg.) 

19.  Heading 

20.  Turn  or  straight 


for  no  broach  and  negative  for  broach  and  had  the  following  classifica¬ 
tion  scores  when  tested  on  a  holdout  set  of  vectors  different  from  the 
set  used  to  generate  the  discriminant: 


Zone 

0 

false 

alarm  rate: 

50% 

Zone 

1 

false 

alarm  rate: 

78% 

Zone 

2 

error 

rate  : 

8.7% 

Zone 

3 

error 

rate  : 

1  .4% 

These  errors  combine  to  make  a  35.6%  combined  prror  rate  (or  error  of 
.356).  Figure  1  shows  a  graph  of  the  confidence  intervals  associated 
with  this  discriminant  function  and  its  error.  For  example,  statistical 
computations  show  there  is  a  .95  probability  that  the  true  combined 
error  for  this  discriminant  lies  in  the  interval  (.356  -  .048  ,  .356  + 
.048)  =  (.308  ,  .404).  That  is,  there  is  a  95%  confidence  that  the  true 
combined  error  for  this  discriminant  is  between  .30  and  .40.  The  confi¬ 
dence  interval  graph  is,  therefore,  an  indication  of  the  reliability  of 
both  the  discriminant  and  the  testing  procedure. 
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Half-width  of  Confidence  Interval 


At  this  point  it  should  be  noted  that,  during  the  course  of  this 


analysis.  Zone  1  broaches  were  not  classed  in  the  same  undesirable 
category  as  those  in  Zones  2  and  3.  It  was  decided  that  Zone  1  broaches 
were  not  an  entirely  unacceptable  operating  condition  and  so  most  of  the 
analysis  was  directed  at  separating  Zone  0  from  Zones  2  and  3,  regardless 
of  how  Zone  1  was  partitioned.  In  computing  classification  results,  Zone 
1  data  were  considered  essentially  no-broach  and  consequently  false  alarm 
rates,  i.e. ,  the  percentage  of  Zone  1  data  called  broach,  were  listed  for 
Zone  1.  Zone  1  is  by  nature  a  type  of  broach  and  so  it  may  not  be  en¬ 
tirely  undesirable  to  have  a  large  percentage  of  Zone  1  called  broach  by 
the  discriminant  function.  In  light  of  this,  the  above  results  may  be 
interpreted  slightly  differently  in  the  sense  of  separating  Zone  0  from 
Zones  1,  2,  and  3.  In  this  case,  the  complement  of  the  above  Zone  1 
false  alarm  rate  becomes  the  Zone  1  error  rate,  with  the  following 
results: 


Zone 

0 

false 

alarm  rate: 

50% 

Zone 

1 

error 

rate  : 

22% 

Zone 

2 

error 

rate  : 

8.7% 

Zone 

3 

error 

rate 

1.4% 

Figure  2  shows  the  confidence  interval  graph  for  this  case  where  the 
combined  error  now  is  .211. 

For  the  purpose  of  reporting  results,  the  above  discriminant  was 
selected  as  the  14-dimensional  basis  for  comparison  with  other  discrim¬ 
inants.  It  is  important  to  note  how  this  function  is  created.  The 
discriminant  vector  plot  option  in  the  structural  analysis  section  of 
OLPARS  computes  two  orthogonal  vectors  to  define  a  discriminant  vector 
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plane.  This  is  a  two-dimensional  plane  in  which  projected  data,  divided 
into  two  classes,  have  the  properties  that  the  spread  of  points  within  a 
class  is  minimized  and  the  spread  between  classes  is  maximized.  These 
properties  make  this  an  ideal  plane  in  which  to  separate  the  two  classes. 
Figure  3  shows  the  discriminant  vector  plane  display  for  the  14-dimensional 
data  set. 


13 


& 


M*************************CC*************************** 


.1.01 
.1.  I. 

+  1 
31  01 
12  1  1 
121 
1 1 2 
+21 


2  HI 
1+21  + 

+  + 


3  +2  + 

3  12+2+ 
33  +223 
3+  23 
32+ 

3 


##*#**##*##*############*##CC#**#*##**##*)|t**#****##**#** 


DISCRIMINANT  OEC TORS  D  0/D 

MISS IS 4 

DISPLAY  SCALE-  231 . 7754  X- 


X~ AXIS  CENTER 
Y-AXIS  CENTER-.. 


SERIAL  RAGE 
I.  OE  I 
•20 . 63 
•3 1 . 56 


OLPARS  Discriminant  Vector  Plot 
for  14  Dimensional  Case 


14 


There  is  some  separation  of  classes  and,  for  example,  one  can  draw  any 
number  of  lines  separating  (in  general)  Zones  2  and  3  from  Zone  0,  each 
with  a  certain  degree  of  error.  Obviously,  the  zones  are  not  completely 
separable;  there  is  overlap.  Here,  in  the  OLPARS  analysis,  the  user  may 
select  the  OLPARS  generated  Fisher  discriminant  or  position  the  line  (from 
which  the  discriminant  function  is  calculated)  in  the  best  place  for  the 
results  desired  (Figure  4).  What  exactly  is  the  best  place  may  ultimately 
depend  on  the  false  alarm  and  error  rates  achieved  by  the  associated  dis¬ 
criminant  and  these  scores  cannot  be  estimated  until  the  discriminant  is 
tested  against  the  proper  holdout  sets  of  data  in  the  Logic  F,valuation 
Section  of  OLPARS.  Consequently,  there  is  a  degree  of  subjectivity  con¬ 
nected  with  the  placement  of  the  discriminant  plane. 
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The  sensitivity  analysis  procedure  using  OLPARS  consisted  of  obtain- 
ing  a  discriminant  vector  for  each  case  resulting  from  the  current 
n  dimensions  considered  n-1  at  a  time.  For  example,  using  fourteen 
dimensions  results  in  fourteen  thirteen-dimensional  cases.  Each  case 
is  run  separately  on  the  same  analysis  set  (of  data)  and  the  resulting 
discriminant  is  tested  separately  on  the  same  "holdout  set"  (data  not 
used  to  generate  the  discriminant).  In  each  case,  the  separability 
of  the  broach  and  no-broach  zones  is  indicated  by  the  classification 
scores  of  the  resulting  discriminants  and  also  by  visual  observation 
of  the  data  point  scatter  in  the  OLPARS  discriminant  vector  plot. 

Since  the  only  change  in  each  case  is  the  deletion  of  one  measurement 
parameter  and  the  reintroduction  of  another  parameter,  the  separability 
evaluations  are  judgments  on  the  meanlngf illness  or  information  content  of 
the  absent  parameter.  The  worst  separability  would  arise  when  the  most 
meaningful  parameter  is  excluded  from  the  measurement  set  and  the  small¬ 
est  decrease  in  separability  would  result  when  the  least  meaningful 
parameter  is  excluded.  By  carefully  examining  the  separabilities  achiev¬ 
ed  by  the  OLPARS  discriminant  vector  plot  for  each  of  the  fourteen 
thirteen-dimensional  cases,  the  least  meaningful  parameter  can  be  select¬ 
ed  and  eliminated  from  the  measurement  set.  The  procedure  can  be  repeat¬ 
ed  for  each  of  the  thirteen  twelve-dimensional  cases  and  so  on  until  only 
one  parameter  (the  most  significant)  remains.  In  such  a  manner,  a 
ranking  of  the  significance  of  the  measurement  parameters  can  be  achieved. 

Two  of  the  fourteen  measurements  are  the  turn/straight  indicator 
and  the  heading  indicator.  Since  these  are  non-continuous  measurements 
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that  easily  partition  the  data  set,  they  were  set  aside  with  the  inten¬ 
tion  of  developing  separate  discriminants  for  each  of  their  possible 
values.  The  sensitivity  analysis  proceeded  on  the  remaining  twelve-di¬ 
mension  space.  Table  3  shows  the  dimension  reduction  sequence  with  all 
discriminants  and  their  false  alarm  (Zones  0  and  1)  and  error  (Zones  2, 
3)  rates.  In  some  cases,  there  was  little  difference  between  two  para¬ 
meters  and  both  were  dropped  in  one  step.  In  every  case,  parameter 
number  10,  "draft",  seemed  the  most  important.  The  significance  of  the 
draft  parameter  is  substantiated  by  comparisons  of  the  measurement 
parameter  histograms  and  by  the  fact  that  the  errors  for  any  of  the 
multidimensional  discr iminants  are  not  far  from  the  errors  achievable  by 
the  "draft"  histogram  alone. 


TABLE  3  -  SENSITIVITY  ANALYSIS 


(Separate  Zone  0  from  Zones  2,  3 
Analysis  Set=ONE  PAGE,  Test  Set=TESTlPAGE) 


ID 

# 

FALSE  . 

ALARM  RATE 

ERROR 

RATH 

DATE /NO 

DIM. 

MEASUREMENT  NUMBERS 

Zone  0 

Zone  1 

Zone  2  Zone  3 

1/24  #1.1 

12 

1-7, 

10, 

11,  13,  14,  15 

36 

79 

9.4 

6.1 

#1.2 

12 

1-7, 

10, 

11,  13,  14,  15 

27 

60 

17.3 

12.2 

#2 

11 

1-7, 

10, 

11,  13,  14 

34 

63 

14.5 

7.0 

#3 

11 

1-7, 

10, 

11,  13,  15 

33 

63 

14.0 

7.0 

#4 

11 

1-7, 

10, 

11,  14,  15 

44 

72 

8.9 

2.6 

#5 

11 

1-7, 

10, 

13,  14,  15 

30 

67 

10.8 

7.8 

#6.1 

11 

1-7, 

11, 

13,  14,  15 

54 

88 

8.4 

13.0 

#6.2 

11 

1-7, 

11, 

13,  14,  15 

60 

90 

4.7 

7.8 

#7 

11 

1-6, 

10, 

11,  13,  14,  15 

37 

76 

9.4 

8.7 

#8 

11 

1-5, 

7, 

10,  11,  13,  14,  15 

37 

79 

8.4 

6.1 

#9 

11 

1-4, 

6, 

7,  10,  11,  13,  14, 

15 

37 

79 

9.4 

6.1 

#10.1 

11 

1-3, 

5-7 

,  10,  11,  13,  14, 

15 

36 

72 

9.8 

7.0 

#10.2 

11 

1-3, 

5-7 

,  10,  11,  13,  14, 

15 

59 

90 

2.8 

3.5 

#10.3 

11 

1-3, 

5-7 

,  10,  11,  13,  14, 

15 

36 

72 

9.8 

7.8 

#11 

11 

1,  2, 

4- 

7,  10,  11,  13,  14, 

15 

42 

78 

7.5 

4.4 

#12 

11 

1,  3- 

■7, 

10,  11,  13,  14,  15 

37 

74 

12.2 

6.1 

#13 

11 

2-7, 

10, 

11,  13,  14,  15 

44 

83 

11  .2 

2.6 

2/17  #1 

10 

2-7, 

10, 

11 ,  14,  15 

35 

66 

15.9 

5.2 

#2 

9 

2-7, 

10, 

11,  14 

44 

78 

13.1 

4.4 

#3 

9 

2-7, 

10, 

11  ,  15 

45 

80 

11.2 

4.4 

#4 

9 

2-7, 

10, 

14,  15 

41 

70 

15.4 

4.4 

#5 

9 

2-7, 

11, 

14,  15 

63 

86 

16.4 

13.0 

#6 

9 

2-6, 

10, 

11,  14,  15 

35 

64 

18.7 

4.4 

#7 

9 

2-5, 

7, 

10,  11,  14,  15 

44 

76 

11.2 

1  .  7 

#8 

9 

2-4, 

6, 

7,  10,  11,  14,  15 

43 

76 

11.7 

3.5 

#9 

9 

2,  3, 

5- 

7,  10,  11,  14,  15 

4  1 

73 

13.  1 

4.4 

#10 

9 

2,  4- 

•7, 

10,  11,  14,  15 

40 

69 

15.0 

4.4 

#11 

9 

3-7, 

10, 

11  ,  14,  15 

44 

77 

12.2 

2.6 

19 


i 


I 

i 


1 

} 


ID  #  FALSE  ALARM  RATE  ERROR  RATE 


DATE /NO 

DIM. 

MEASUREMENT  NUMBERS 

Zone  0 

Zone  1 

Zone  2 

Zone  3 

2/24 

#1 

9 

2-5,  7,  10,  11, 

14, 

15 

42 

72 

11  .  7 

1.7 

#2 

8 

2-5,  7,  10,  11, 

14 

44 

77 

12.6 

2.6 

#3 

8 

2-5,  7,  10,  11, 

15 

40 

71 

13.6 

2.6 

#4 

8 

2-5,  7,  10,  14, 

15 

40 

71 

14.0 

1.7 

#5 

8 

2-5,  7,  11,  14, 

15 

80 

92 

10.3 

8.7 

#6 

8 

2-5,  10,  11,  14, 

15 

44 

75 

13.1 

1.7 

#7 

8 

2-4,  7,  10,  11, 

14, 

15 

48 

81 

9.4 

1.7 

#8 

8 

2,  3,  5,  7,  10, 

11, 

14, 

15 

45 

76 

9.8 

1 . 7 

#9 

8 

2,  4,  5,  7,  10, 

11, 

14, 

15 

49 

81 

7.9 

1.7 

#10 

8 

3-5,  7,  10,  11, 

14, 

15 

43 

76 

11.2 

1.7 

2/27 

#1 

7 

3,  4,  5,  7,  10, 

14, 

15 

41 

71 

13.6 

1.7 

#2 

6 

3,  4,  5,  7,  10, 

14 

45 

79 

11.7 

1.7 

#3 

6 

3,  4,  5,  7,  10, 

15, 

50 

80 

12.6 

4.4 

#4 

6 

3,  4,  5,  7,  14, 

15 

57 

64 

39.7 

33.0 

#5.1 

6 

3,  4,  5,  10,  14, 

15 

44 

74 

13.5 

1.7 

#5.2 

6 

3,  4,  5,  10,  14, 

15 

42 

72 

13.1 

3.5 

#6 

6 

3,  4,  7,  10,  14, 

15 

43 

75 

13.1 

1.7 

#7 

6 

3,  5,  7,  10,  14, 

15 

42 

72 

12.6 

1.7 

#8 

6 

4,  5,  7,  10,  14, 

15 

49 

86 

7.9 

1.7 

2/28 

#1.1 

5 

4,  7,  10,  14,  15 

48 

85 

7.0 

1.7 

#1.2 

5 

4,  7,  10,  14,  15 

47 

82 

10.1 

2.6 

#2 

4 

4,  7,  10,  14 

46 

80 

10.3 

2.6 

#3 

4 

4,  7,  10,  15 

42 

74 

17.3 

3.5 

#4 

4 

4,  7,  14,  15 

76 

90 

10.3 

7.8 

#5 

4 

4,  10,  14,  15 

34 

59 

20.1 

5.2 

#6 

4 

7,  10,  14,  15 

45 

77 

11.2 

1.7 

3/1 

#1 

4 

7,  10,  14,  15 

41 

74 

13.5 

2.6 

20 


ID 

DATE /NO 

# 

DIM. 

MEASUREMENT  NUMBERS 

FALSE  ALARM  RATE 
Zone  0  Zone  1 

ERROR 

Zone  2 

RATE 

Zone 

#2 

3 

7,  10,  14 

41 

72 

13.5 

4.4 

#3 

3 

7,  10,  15 

40 

70 

16.4 

2.6 

#4 

3 

7,  14,  15 

68 

86 

15.4 

12.2 

#5 

3 

10,  14,  15 

28 

50 

22.0 

5.2 

3/1.1  #1 

3 

10,  14,  15 

27 

48 

24 . 7 

6.  1 

#2 

2 

10,  14 

34 

61 

24.3 

6.1 

#3 

2 

10,  15 

29 

54 

21  .5 

5.2 

#' 4 

2 

14,15  no  apparent 

separation  x 

X 

X 

X 

H  -  #1 

1 

10 

23 

43 

28.0 

1  I  .0 

in 

I 

10 

33 

61 

19.0 

6.0 

it  3 

I 

10 

48 

82 

10.0 

3.0 

21 


The  order  in  which  the  parameters  were  dropped  in  the  dimension 
reduction  process  was 

1.  Speed  (1),  total  fan  air  flow  (13) 

2.  Roll  rate  (6) 

3.  Pitch  angle  (2),  stern  seal  position  (11) 

4.  Pitch  rate  (3),  roll  angle  (5) 

5.  Yaw  rate  (4) 

6.  Cushion  pressure  (7) 

7.  Port  and  starboard  nozzle  angles  (14,  15) 

At  the  top  of  this  list  are  the  least  meaningful  parameters  for  achieving 
separability.  The  least  significant  were  found  to  be  "speed"  and  "total 
fan  air  flow",  both  of  which  were  expected  to  have  greater  effect  on 
the  decision  process. 

Table  4  shows  the  best  (in  a  subjective  sense)  results  from  indi¬ 
vidual  discriminants  at  each  stage  of  dimensional  reduction.  This  table 
will  give  some  impression  of  the  trade-offs  in  separability  that  are 
encountered  in  a  process  of  this  kind.  Unfortunately,  although  these 
functions  are  the  best  among  many  considered,  they  need  not  be  best  in 
the  sense  of  being  most  suited  for  the  operational  efficiency  of  the 
craft.  For  example,  it  may  be  operationally  beneficial  to  shift  the 
discriminant  to  gain  better  Zone  2  error  rates  at  the  expense  of  more 
false  alarms.  This  question  has  not  been  addressed  by  this  study. 
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TABLE  4  -  "BEST"  DISCRIMINANTS  IN  DIMENSIONAL 
REDUCTION  PROCESS 


# 

DIM. 

FUNCTION 

ID 

FALSE  ALARM  RATE 

ERROR  RATE 

Zone  0 

Zone  1 

Zone  2 

Zone  3 

20 

11-18-5 

7.9 

45 

19.6 

5.6 

19 

3-9-4 

12 

36 

25.7 

20.9 

14 

12-14-10 

50 

78 

8.7 

1  .4 

12 

1-24-1.2 

27 

60 

17.3 

12.2 

11 

1-24-5 

30 

67 

10.8 

7.8 

10 

2-17-1 

35 

66 

15.9 

5.2 

9 

2-17-6 

35 

64 

18.  7 

4.4 

8 

2-24-4 

40 

71 

14.0 

1.7 

7 

2-27-1 

41 

71 

13.6 

1.7 

6 

2-27-7 

42 

72 

12.6 

1  .  7 

5 

2-28-1 . 1 

48 

85 

7.0 

1.7 

4 

2-28-5 

34 

59 

20.1 

5.2 

3 

3-1. 1-1 

27 

48 

24. 7 

6.  1 

2 

3-1. 1-3 

29 

54 

21  .5 

5.2 

1 

H-l 

23 

43 

28.0 

11.0 

23 


The  tabulated  results  for  the  five-dimensional  case  show  very 
little  degradation  from  those  of  the  fourteen-dimension  results.  This 
suggests  that  the  measurements  deleted  to  obtain  the  five  dimensions  have 
little  significant  effect  on  the  classification  performance  of  the 
system.  Likewise,  the  results  for  the  two  dimensions  agree  closely  with 
the  results  for  four  dimensions.  Table  4  contains  results  for  one 
discriminant  not  based  on  the  "draft"  measurement  parameter  and  that  is 
the  19-dimension  case.  Comparison  of  this  case  with  the  twenty-dimension 
case  gives  an  indication  of  the  degradation  induced  by  ignoring  the 
"draft"  parameter.  The  results  from  the  twenty-dimension  case  must  be 
qualified  since  they  are  achieved  by  including  the  six  parameters  judged 
to  measure  only  supplemental  effects  of  broach  and  not  valid,  controll¬ 
able  phenomena  for  prediction. 

As  a  basis  for  further  comparison,  the  confidence  interval  graph 
for  the  two-dimensional  discriminant  listed  in  Table  4  is  shown  in  Figure 
5.  The  combined  error  for  separating  Zones  0  and  1  from  Zones  2  and  3  is 
.281.  The  confidence  interval  graph  indicates  a  slightly  greater  confi¬ 
dence  in  the  accuracy  of  this  error  estimate  than  in  the  estimate  for  the 
fourteen-dimensional  discriminant  (Figure  1).  The  graph,  therefore, 
supports  the  conclusion  that  there  has  been  little  degradation  in  dis¬ 
criminating  power  from  fourteen  dimensions  to  two  dimensions. 
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It  is  important  to  note  that  discriminants  with  very  low  error 
rates  (for  Zone  3)  on  this  data  set  have  the  higher  false  alarm  rates 
(for  Zone  0).  This  results  from  the  inherent  overlap  of  Zone  3  and  Zone 


0  data  points  and  the  placement  of  the  discriminant  plane.  In  general, 
during  the  OLPAJRS  analysis,  the  planes  were  drawn  to  obtain  the  best 
error  rates  without  causing  great  increases  in  false  alarm  rates  for 
each  Zone  3  point  added  to  the  proper  side  of  the  plane.  A  more  balanced 
set  of  error  and  false  alarm  rates  can  always  be  achieved  by  positioning 
Lhe  discriminant  plane  more  towards  the  center  of  the  overlap  in  the 
OLPARS  discriminant  vector  plot  display.  (See  Figure  4) 

MULTIPLE  DISCRIMINANTS 

The  two  parameters  deleted  prior  to  the  OLPARS  analysis  of  twelve 
dimensions  were  the  "heading"  and  "turn/straight"  indicators.  These 
were  set  aside  because  of  the  non-cont inuous  nature  of  the  arbitrarily 
assigned  numeric  values  and  the  ease  with  which  they  lent  themselves 
to  the  use  of  multiple  discriminants.  Table  3  lists  scores  for  discrim¬ 
inants  created  by,  in  one  case,  sorting  the  data  set  between  "turn"  data 
and  "straight"  data  and,  in  the  other  case,  sorting  according  to  "head¬ 
ing".  The  principal  data  sets  obtained  from  the  heading  parameter  were 
those  for  "calm",  "head",  and  "following"  conditions.  Some  measurements 
existed  in  the  data  set  for  "quarter"  and  "beam"  seas,  but  it  was  felt 
these  sets  were  not  large  enough  to  give  reliable  results.  When  results 
are  compared  with  those  achieved  by  the  twelve-dimensional  discriminant 
of  Table  4,  and  when  it  is  assumed  that  results  for  "quarter"  and  "beam" 
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data  are  not  significantly  different,  it  can  be  seen  that  a  set  of 
discriminants  based  on  heading  can  improve  error  rates  slightly.  Sep¬ 
arate  discriminants  for  "turn"  and  "straight”,  on  the  other  hand,  do  not 
have  much  effect.  This  may  be  due  to  the  low  sea  states  of  all  points 
in  this  data  set;  sea  states  never  exceeded  sea  state  two.  It  seems 
reasonable  to  expect  the  distinction  between  "turn"  and  "straight"  to 
have  more  significance  in  higher  sea  states. 

Figure  6  shows  the  confidence  interval  graph  for  the  composite 
discriminant  described  in  A. 1  of  Table  5.  The  total  composite  error  is 
.261  and  the  graph  of  confidence  intervals  for  this  case  is  very  similar 
to  that  for  the  two  dimensional  discriminant  shown  in  Figure  5. 

TABLE  5.  MULTIPLE  DISCRIMINANTS  BASED  ON  HEADING 
AND  TURN  STRAIGHT 

A.  Composite  results  separating  Zones  0,  1  from  Zones  2,  3  using  sep¬ 
arate  discriminants  for  HEAD,  FOLLOW,  and  CALM  conditions  of  heading 
parameter. 

1.  Including  Turn/Straight  measurement  (13  dimensions): 


False 

Alarm 

Error 

Zone  0 

Zone  1 

Zone 

2  Zone  3 

HEAD 

CO 

CM 

• 

.60 

.16 

.044 

FOLLOW 

.22 

.57 

.22 

.25 

*CALM 

.20 

.37 

.062 

.00 

COMPOSITE 

.24 

.51 

.16 

.11 

*  Note:  CALM  results  are  the  same  as  in  A. 2.  since  there  is  no  Turn  data 
for  CALM. 
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2.  Not  including  Turn/Straight  measurement  (12  dimension): 


_ False  Alarm _ Error _ 

_ Zone  0  Zone  1 _ Zone  2 _ Zone  3 

HEAD  .13  .42  .28  .13 

FOLLOW  .35  .69  .16  .  .19 

*CALM _ ._20 _ 137 _ .062 _ .00 

COMPOSITE  .22 _ .47 _ J_9 _ .13 

*  Note:  CALM  results  are  the  same  as  in  A.l.  since  there  is  no  Turn 
data  for  CALM. 

B.  Composite  results  for  separating  Zones  0,  1  from  Zone  2,  3  using 
separate  discriminants  for  HEAD,  FOLLOW,  and  CALM  but  with  Zone  1 
data  not  used  in  constructing  discriminants. 

1.  Including  Turn/Straight  (13  dimension) 


False  Alarm 


Error 


_ Zone  0 _ Zone  1 _ Zone  2 _ Zone  3 

HEAD  .31  .61  .12  .067 

FOLLOW  .26  .88  .12  .17 

CALM _ ^23 _ 137 _ ;_031 _ .  10 

COMPOSITE  .27  .60  .10  .11 
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2.  Not  including  Turn/Straight  (12  dimensions) 


False 

Alarm 

Error 

Zone  0 

Zone  1 

Zone 

2  Zone  3 

HEAD 

.33 

.65 

.12 

.067 

FOLLOW 

.29 

.86 

.12 

.11 

CALM 

.23 

.37 

.031 

.  10 

COMPOSITE  .28 _ ._61 _ .JO _ .089 

C.  Composite  results  separating  Zones  0,  1  from  2,  3  using  separate 

discr iminants  for  Turn  and  Straight.  (Zone  1  data  not  used  in  con¬ 
structing  discriminants) 

1.  Including  heading  measurement  parameter  (13  dimensions) 


False 

Alarm 

Error 

Zone  0 

Zone 

1 

Zone 

2  Zone  3 

TURN 

.40 

.82 

.11 

.15 

STRAIGHT 

.37 

.64 

.098 

.084 

COMPOSITE 

.37 

.66 

.10 

.10 

it  including 

heading  i 

measurement 

(12  dimensions) 

False 

Alarm 

Error 

Zone  0 

Zone 

1 

Zone 

2  Zone  3 

TURN 

.32 

.59 

.25 

.  1  5 

STRAIGHT 

.30 

.58 

.13 

.084 

COMPOSITE 

.30 

.58 

.16 

.10 
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Expected  test  set  error  rate  (interval  center)  =  .261 


Fig.  6  Confidence  Interval  Graph 
(13  Dimensions  Composite) 
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CONCLUSIONS 


This  report  has  mentioned  only  a  few  of  the  discriminant  analysis 
results  achieved  with  this  broach  data  set-  Appendixes  A  and  R  contain  lists 
of  discriminant  functions  obtained  from  various  analyses  and  the  classification 
results  achieved  on  complementary  test  sets.  In  spite  of  this  mass  of  statistics, 
the  sensitivity  study  conducted  in  this  project  was  hv  no  means  exhaustive;  it 
would  not  be  practical  to  consider  all  subsets  of  the  fourteen  measurements. 
Furthermore,  the  method  of  training  and  testing  is  not  optimal  in  the  sense 
that,  in  general,  only  one  part  of  the  data  set  was  used  for  generating  the 
discriminants  and  the  remaining  portion  was  used  for  testing.  A  more  elaborate 
scheme  for  computing  the  discriminant  from  all  but  one  point  in  the  data  set, 
testing  on  the  one  holdout  point  and  repeating  until  each  point  is  used  once  as 
a  holdout  would  provide  statistically  more  valid  discrimination  results  hut 
require  too  much  time  to  perform.  The  results  obtained  in  the  present  analysis, 
when  properly  considered  within  the  test  conditions  used  and  with  the  associated 
confidence  intervals,  do  support  the  following  conclusion:  Rroach  is  separable 
from  no-broach  with  low  error  and  moderate  false  alarm  rates.  Draft,  as  might 
be  expected,  is  a  significant  measurement  parameter,  and  heading,  when  used 
in  a  multiple  discriminant  context,  offers  improved  separability.  No  measurement 
parameter  other  than  draft  exhibited  any  clearly  greater  effect  on  results  than 
any  other  parameter.  However,  this  analysis  shows  that  sets  of  other  parameters 
(which  are  more  easily  measured  and  controlled  than  draft)  can  recognize  broach 
nearly  as  well  as  the  draft  measurement.  Finally,  all  results  and  conclusions, 
strictly  speaking,  apply  only  to  the  operating  conditions  during  the  given 
tests.  Among  these  conditions,  low  sea  states  and  unevenly  distributed  speed 
data  are  the  most  notable. 
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APPENDIX  A 


DISCRIMINANT  FUNCTIONS 

The  following  pages  contain  a  computer  generated  list  of  the  150 
discriminant  functions  (hyperplanes)  considered  in  the  broaching 
analysis.  The  planes  are  considered  to  be  of  the  form 

alXl  +  a2x2  +...+  a20x20  +  a2i  =  0 

where  the  a^'s  are  the  coefficients  of  the  function.  In  the  listing  that 
follows,  the  function  number  is  given  and  the  twenty-one  coefficients 
describing  that  function  are  listed  below  the  function  number.  The  first 
twenty  coefficients  correspond  to  the  twenty  measurement  parameters  of  the 
initial  space .  If  a  measurement  is  not  being  considered  its  corresponding 
coefficient  value  is  zero.  For  example,  function  number  thirty  may  be 
considered  a  function  in  1 4-dimensional  space  operating  on  measurements 
1-7,  10,  11,  13,  14,15,  19,  and  20. 
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APPENDIX  B 

CLASSIFICATION  RESULTS  FOR  DISCRIMINANT  FUNCTIONS 

Discriminant  functions  were  typically  generated  by  OLPARS  analysis  of 
data  composing  only  a  subset  of  the  whole  data  set,  thereby  allowing  a 
complementary  set  of  data  with  which  to  statistically  test  the  accuracy  of 
the  discriminants.  The  following  tables  refer  to  these  data  sets  by  name. 
The  composition  of  these  sets  is  described  in  a  separate  table  at  the 
end  of  this  appendix. 

The  numbers  listed  for  false  alarm  and  error  rates  are  in  terras  of 
percentages.  For  example,  the  Zone  0  false  alarm  rate  for  plane  number 
one  is  given  as  the  number,  5.  This  means  that  5%  of  Zone  0  was  classed 
as  broach  for  a  false  alarm  rate  of  5%  or  .05. 
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PLANE  ANALYSIS  TEST  #  MEASUREMENTS  FALSE  ALARM  RATES _ ERROR  RATES 

NO.  ID  SET  SET  DIM  USED  ZONE  0  ZONE  1  ZONE  2  ZONE  3 
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Description  of  Data  Sets 


NAME 

DESCRIPTION 

ZoneO 

Zonel 

COUNTS 

Zone 2  Zone 3 

TOTAL 

WHOLE  SET 

Whole  data  set 

152, 

200, 

276, 

147; 

775 

DATASET 1 : 

Concatenate  Zone  0,  1,  2, 
Select  odd  If'e d  vectors 

3, 

76, 

100, 

138, 

74; 

380 

DATASET2: 

Concatenate  Zone  0,  1,  2, 
Select  even  tf'ed  vectors 

3, 

76, 

100, 

138, 

73; 

387 

ONEPAGE: 

Contains  every  4th  vector  of 
ZoneO,  every  4th  of  Zonel, 
every  4th  of  Zone 2,  and  every 
4th  of  Zone  3 

33, 

44, 

62, 

32; 

171 

TEST 1 PAGE: 

Complement  of  ONEPAGE 
within  whole  data  set 

119, 

156, 

214, 

115; 

604 

P1STATES12 : 

Contains  every  4th  vector 
of  STATES  1 2  starting  with 
1st  one 

26, 

34, 

58, 

31; 

149 

STATES 12: 

Contains  all  data  points 
with  sea  state-l  or  2 

101, 

138, 

233, 

121; 

593 

CALM: 

All  points  with  sea  state* 
"calm" 

51, 

62, 

43, 

26; 

182 

HEAD1P: 

Concatenate  ZoneO,  Zonel, 

Zone 2,  Zone  3 

Select  every  4th  vector  with 
paraml9,  "heading,"  set  to 
1-HEAD 

13, 

18, 

26, 

13; 

72 

FOLLOW1P: 

Same  as  HEAD1P  but  select 
every  4th  vector  with 
#19-3-"follow" 

10, 

12, 

22, 

12; 

56 

CALM1P: 

Same  as  HEAD1P  but  select 

12, 

16, 

11, 

6; 

45 

every  4th  w/#  19-5-"calm" 


54 


NAME 

DESCRIPTION 

ZoneO 

Zonel 

COUNTS 

Zone2  Zone3 

TOTAL 

TURNIP: 

Same  as  HEAD1P  but  select 
every  4th  vector  w/#20=l= 
"turn" 

6, 

6, 

18, 

9; 

39 

STRIP: 

Same  as  HEAD1P  but  select 
every  4th  w/#20=2="straight" 

31, 

45, 

50, 

28; 

154 

HEADR: 

Complement  of  HEAD1P  within 
#19="head" 

39, 

57, 

76, 

45; 

21  7 

FOLLOW 1 R : 

Complement  of  FOLLOW1P  within 
#19="fol low" 

31, 

35, 

68, 

36; 

170 

CALMR : 

Complement  of  CALM1P  within 
#19="calm" 

39, 

46, 

32, 

20; 

137 

TURNR: 

Complement  of  TURNIP  within 
#20="turn" 

19, 

17, 

55, 

27; 

118 

STRR : 

Complement  of  STRIP  within 
#20="straight" 

96, 

132, 

153, 

83; 

464 

ZONES03 

ZoneO  and  Zone3  data 

152, 

o. 

0, 

147; 

299 
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INITIAL  DISTRIBUTION 


Copies 

2  NAVSEA 

I  -  Code  03R3  (J.  Cuthbert) 
1  -  Code  3213  (R.  Reane) 

1  DLSIE 

12  DTIC 

CENTER  DISTRIBUTION 


Copies 

Code 

Name 

5 

PMS  304-221 

W.  White 

1 

15 

W.  Morgan 

1 

1576 

W.  Smith 

1 

1630 

A.G.  Ford 

1 

1630 

Bell  Aerospace 

1 

1630 

Rohr  Marine,  Inc. 

1 

1630 

Hydronautics 

1 

16 

H.  Chaplin 

1 

1800 

G.  H.  Gleissner 

1 

1809.3 

D.  Harris 

1 

182 

A.  Camara 

30 

1824 

J.  Garner 

1 

184 

J.  Schot 

1 

185 

T.  Corln 

1 

187 

M.  Zubkoff 

1 

522. 1 

Unclassified  Library  (C) 

1 

522.2 

Unclassified  Library  (A) 

57 


HOCEDDO  MOB  MJUK-MOT  FILMED 


I 


DTNSRDC  ISSUES  THREE  TYPES  OF  REPORTS 

1.  DTNSRDC  REPORTS.  A  FORMAL  SERIES.  CONTAIN  INFORMATION  OF  PERMANENT  TECH 
NICAL  VALUE.  THEY  CARRY  A  CONSECUTIVE  NUMERICAL  IDENTIFICATION  REGARDLESS  OF 
THEIR  CLASSIFICATION  OR  THE  ORIGINATING  DEPARTMENT. 

2.  DEPARTMENTAL  REPORTS.  A  SEMIFORMAL  SERIES,  CONTAIN  INFORMATION  OF  A  PRELIM 
INARY,  TEMPORARY.  OR  PROPRIETARY  NATURE  OR  OF  LIMITED  INTEREST  OR  SIGNIFICANCE 
THEY  CARRY  A  DEPARTMENTAL  ALPHANUMERICAL  IDENTIFICATION. 

3.  TECHNICAL  MEMORANDA.  AN  INFORMAL  SERIES,  CONTAIN  TECHNICAL  DOCUMENTATION 
OF  LIMITED  USE  AND  INTEREST.  THEY  ARE  PRIMARILY  WORKING  PAPERS  INTENDED  FOR  IN 
TERNAL  USE.  THEY  CARRY  AN  IDENTIFYING  NUMBER  WHICH  INDICATES  THEIR  TYPE  AND  THE 
NUMERICAL  CODE  OF  THE  ORIGINATING  DEPARTMENT.  ANY  DISTRIBUTION  OUTSIDE  DTNSRDC 
MUST  BE  APPROVED  BY  THE  HEAD  OF  THE  ORIGINATING  DEPARTMENT  ON  A  CASE  BY  CASE 
BASIS. 


